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ABSTRACT 

There has been a marked increase in the use of online learning over the past decade. 
There remains conflict in the current body of research on the efficacy of online versus face to 
face learning in these environments. One resolution of these issues is the hybrid learning option 
which is a combination of face-to-face classroom instruction with asynchronous online elements. 
The hybrid option has been gaining in popularity as it is capable of combining the advantages of 
traditional face-to-face and online learning environments. In this paper, we seek to evaluate the 
effectiveness of hybrid instructional programs by comparing the use of hybrid learning against 
traditional classroom instruction. End of term grades are used to measure student retention of 
course material and are compared across the two groups. We find that the final grades for the 
students in the hybrid situation were significantly greater than those earned in the traditional 
format, all other aspects of the course remaining the same. 
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INTRODUCTION 

There has been a marked increase in the use of online learning over the past decade. The 
2007 Sloan survey of online learning found that in the fall of 2006, 3.5 million higher education 
students took online courses. This was an increase of more than 100% since 2002 when 
enrollment in online courses was 1.6 million and more recent data supports this with over 5.6 
million students during the fall term of 2009 enrolling in online courses (Allen & Seaman 2010). 
The increase in the use of online learning has been caused by increases in the educational uses of 
technology. Advances in internet technology, including expanded broadband penetration, allow 
a wider range of online learning tools to be available and accessible to a larger audience (for 
example, course management systems, online testing and exam tools, electronic books, 
simulations, text messaging, podcasting, wikis, blogs). Among the advantages for promoting 
online educational options are increased flexibility, elimination of geographical barriers, 
improved convenience, and effectiveness for individualized and collaborative learning (Wu et 
al„ 2010) 

Most of the current body of research finds that learning outcomes are much the same in a 
completely online or a traditional face-to-face classroom environment (Anstine & Skidmore, 
2005; Schulman & Sims, 1999; Russell, 1999; Terry, 2003). Although there is disagreement 
among academics regarding the efficacy of online course delivery as compared with traditional 
face-to-face delivery, the use of online delivery is moving forward as a result of the benefits to 
students taking the courses and the institutions delivering them. Graduates must be engaged, 
self-directed learners to succeed in their chosen fields and this skill is developed in an online 
format. An online format can reduce the variability in course delivery across instructors and can 
also increase the number of students who can access higher education. Finally, ongoing delivery 
of online courses can be very cost effective for the institution and for the student. 

As the tools and expertise associated with the provision of online learning improves, so 
does the quality of the educational experience and the satisfaction of students who engage in 
online learning. Many students identify the flexibility in scheduling and instructional pace as 
key components of a positive online experience (Alexandra, 1996). In the 2011 National Online 
Learners Priorities Report which has data based on more than 99,000 student responses from 108 
institutions between the fall of 2008 and the spring of 2011, students cite convenience as the 
number-one reason they are enrolling in online programs. That reason was followed closely by 
flexible pacing for the program and work schedules. 

Along with the positive viewpoints regarding online learning come some identified areas 
for improvement. Nemanich, et.al, (2009) find that student enjoyment of the course is 
positively associated with learning outcomes in a traditional classroom but not online, and 
student ability is positively associated with learning outcomes in an online format but not in a 
traditional classroom. The Educause Center for Applied Research (ECAR) study of 
undergraduate students and information technology find that students enjoy taking online classes 
but they want more face to face interaction (Regan, 2007). In a study of nursing students, 
decreased (physical) contact with faculty and peers was a disadvantage in web-based learning 
(Halstead, 2000). 

One resolution of these issues is the hybrid learning option which is a combination of 
face-to-face classroom instruction with asynchronous online elements. The hybrid option has 
been gaining in popularity as it is capable of combining the advantages of traditional face-to-face 
and online learning environments. Even though there has recently been increased interest 
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regarding the use of this hybrid option (Kumrow, 2007; Tabor, 2007; Hall, 2006, Bates & 
Watson, 2008), the majority of the research has focused on the comparison of online 
versus traditional face-to-face instruction. In studies where the hybrid option has been 
tested, mixed reviews in knowledge gain, or student satisfaction are evident between all 
three types of instruction; traditional face to face, online or the hybrid option (Reasons, 
2004). In one such study, Banerjee (2011) reported over half of the respondents (fifty- 
seven percent) enjoyed a blended format in course delivery. On the other side, Castle and 
McGuire (2010) suggested that "undergraduate and graduate students across various 
disciples generally prefer onsite learning to either online or hybrid teaching modalities." 
(p. 37). Apparently college students perceive online courses to require more work, and 
this can reduce the satisfaction they have with both online and hybrid learning. 

The ability of students to understand and apply the knowledge they are presented 
with is a key indicator of whether or not an instruction methodology is successful. 
Although it can be difficult to evaluate whether a particular course in a college degree 
program has added value to student learning, there is some evidence that learning 
effectiveness can be evaluated by a comparison across groups of students who have 
received instruction via two different methodologies but are assessed in the same manner 
(Anstine & Skidmore, 2005). In a recent study where physical therapist educators 
“flipped” the classroom, incorporating the use of technology to deliver less active events 
(listening to lectures and accessing links to resources) outside of traditional class time, 
they received positive feedback from the students involved in this experience. 
Furthermore the blended environment enhanced learning outcomes as measured by 
course grades (Boucher et. al. 2013). 

TRADITIONAL FACE-TO-FACE INSTRUCTION 

Campus based or traditional instruction is the easiest for us to understand 
and it is the format that students and faculty are the most familiar with as it has 
been the common method of teaching for many decades. It is characterized by 
student and faculty interaction via lectures, discussion and exams on campus at 
scheduled times of day. Professors hold office hours and students can ask 
questions during classes. We often perceive that the interaction between 
professor and students associated with the face-to-face format enables a high 
quality of learning, but many studies dispute this claim and find online learning 
facilitates higher levels of retention when compared with traditional lectures 
(Johne, 2003; Vasarhelyi, 1997; Shachar & Neumann, 2007). 

Another perceived weakness of face-to-face instruction is lack of time and 
place flexibility. For working professionals or non-traditional students, the 
requirement to conduct classes on campus can negate their ability to acquire 
further education or cause them to have to miss class sessions which decreases 
their learning. With the rising cost of education, even traditional students may 
struggle to attend face-to-face classes due to time commitments associated with 
part-time jobs or stringent requirements involved in effective maintenance of non- 
academic scholarships. These factors necessarily contribute to the increase in 
demand for online learning formats. 
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ONLINE LEARNING 

The identified challenges associated with online learning are the requirement for students 
to have more time management skills in order to be sufficiently self-directed to complete the 
course and the fact that topics not covered in the face to face portion of the class are perceived to 
be of lesser value (Tabor, 2007). In addition, the most significant complaints directed at online 
courses are the lack of student-teacher interaction (Fann, 2001) and the loss of a sense of 
community (James, 2001). 

A key benefit of online and hybrid learning is the wide variety of learning methods and 
tools that are supported, allowing each individual student to achieve a more customized approach 
to learning. In particular, student groups formed in person appear to interact more effectively 
during their online communication sessions than when they are communicating face-to-face and 
fewer physical meetings represent less travel time for commuting students and an attractive 
alternative for non-traditional working students (Tabor, 2007). Since it has been recognized that 
adults learn at different paces (Kasworm, 2003), including self-paced instruction as part of the 
course structure can enhance the learning experience. In addition other factors may influence the 
success of online learning. La Bay and Comm (2011) contend that student satisfaction with 
online delivery is also impacted by academic status (graduate versus undergraduate), gender, and 
students' inclination to take online courses. 

THE HYBRID LEARNING OPTION 

Hybrid learning involves a blend between face-to-face and online learning where the mix 
between classroom and online instruction can vary based on consideration of differences in 
course content and the level of student comfort with online learning. The decision regarding the 
amount of online versus classroom instruction is at the discretion of the instructor and is very 
dependent on the course content, but hybrid courses typically require students to meet face-to- 
face approximately 50% of the time and utilize a course management system such as Blackboard 
for the remainder of the course requirements. The hybrid model allows working professionals or 
busy graduate students an opportunity to reduce their in-class time yet maintain an effective 
amount of contact with faculty and peers. The hybrid model is thought to be one of the most 
effective new education strategies as it can capture the best aspects of online and face-to-face 
classroom instruction (Skill, 2002). 

Educators can choose to implement the hybrid option across a wide range of situations, 
making it an excellent candidate for adult education and one that can grow and evolve along with 
advancements in technology. This being the case, we must seek to understand any learning 
retention benefits associated with the partial online model. While research and theory about the 
hybrid option is growing, conflicting results are often reported about the quality of learning 
outcomes between it, online and face-to-face instruction (Fjermestad, 2005, Pinto and Anderson, 
2013). 

In this paper, we seek to evaluate the effectiveness of hybrid instructional programs by 
comparing the use of hybrid learning against traditional classroom instruction. End of term 
grades are used to measure student retention of course material and are compared across the two 
groups. We find that the final grades for the students in the hybrid situation were significantly 
greater than those earned in the traditional format, all other aspects of the course remaining the 
same. In addition we provide anecdotal evidence of student satisfaction with the partial on-line 
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structure obtained from an in-class survey and offer some student commentary on how 
the on-line portion of the course aided their ability to learn and retain course material. 

METHODOLOGY 

The nonrandom sample for this study included students enrolled in three sections 
of an undergraduate-level introductory management course offered at the school of 
business of a small private university located in the Mid-Atlantic region. The course is 
designed to introduce students to the management function and prepare them for further 
study within the management major. It is a required course for every undergraduate 
business major. One section of the course was offered in the fall of 2006 and the others 
were offered in the fall of 2007. All classes were scheduled between 10am and 1pm and 
taught by the same professor. 

One section of the course was conducted using a traditional face-to-face format 
that included lecture and discussion and had a class enrollment of 50 students. The other 
two sections were structured using a web-based hybrid format in which 60% of the class 
was conducted using the traditional face-to-face format and the remaining 40% was 
conducted online. The online portion consisted of available lecture and support material, 
along with weekly assignments and quizzes. Tests were all during class time. The hybrid 
class enrollment was 25 students for each section, for a total of 50 students. The size of 
the total sample is 100, with N = 50 participating in the traditional face-to-face classroom 
section and N = 50 participating in the hybrid sections. 

All three sections covered the same material and required students to complete 
regular assignments and quizzes. The traditional face-to-face structure required students 
to hand in hard copies of their assignments and quizzes for grading while the hybrid class 
required students to use an electronic drop box to submit their work to the professor. All 
sections had three in-class exams which consisted of multiple choice questions. Even 
though the questions across all assignments, tests and quizzes were not identical, they 
were structured to include an equal representation of questions based on comprehension 
difficulty (as defined by Blooms taxonomy) across the levels of easy, medium and 
difficult. The test questions were all generated from an online test generator that 
identified the comprehension difficulty so that all exams were equivalent in terms of 
difficulty level. 

RESULTS 

The gender distribution of all study participants was an even mix between male 
and female with 52% (n = 52) of enrolled students being female and the remaining 48% 

(n = 48) being male. The majority of the participants were first or second year students 
from the school of business; students from other schools within the university represented 
17% (n = 17) of the population. Within each group the mix of students from within and 
outside the business school was consistent. 

The outcome variable for this study was academic achievement as measured by 
end of course grades. To determine which course format yielded higher achievement 
scores for students, a t-test was performed with end-of-course grades by group. Data 
analysis revealed that students in the hybrid sections had significantly higher end-of- 
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course grades (t = 2.553, df = 98, p = 0.12). The average for this group was 0.831 (SD = 0.095) 
and the average for the lecture section was 0.782 (SD = 0.097). Table 1 summarizes these result 
(Appendix). 

In order to control for the possibility that the difference in end of course grades could be 
due to differences in the academic ability of the students, we compared GPA scores between 
those enrolled in the hybrid classes versus those students enrolled in the traditional face-to-face 
format. There was no significant difference between the two groups (t = 0.522, df = 98, p 
=0.248) with the average for the hybrid section being 2.96 (SD = 0.556 ) versus the face-to-face 
section which was 3.02 (SD = 0.523). Table 2 provides a summary of these results (Appendix)’ 
A three question anonymous survey was administered near the end of the term to the 50 
students who participated in the hybrid sections. Participation was voluntary and thirty-four of 
the 50 participants (n = 34) completed the survey for a 68% response rate. Question 1, with a 5 
point Likert scale, questioned students’ enjoyment of the online portion of the course and 29 (n = 
29) students reported enjoying all or most of the online aspects of the course. Question 2 asked 
for a yes or no choice regarding whether students felt the online portion of the class improved 
their learning and retention of the course material. On Question 2, 31 (n = 31) students agreed 
that the online portion of the course assisted their learning and retention. The third question was 
open ended, allowing for participants to freely provide comments regarding how the online 
portion did or did not improve their ability to learn the course material. The researcher 
distributed the surveys in the last 15 minutes of the last class of the semester prior to the final 
exam. Table 3 provides examples of these responses (Appendix). 

PEDAGOGICAL IMPLICATIONS 

This study supports the idea that the use of an online component in a course can enhance 
student learning outcomes. In order for students to be successful in this format, extensive up 
front planning must be done by the instructor. Students must be provided with clear and 
unambiguous information regarding the components of the course which are online and with the 
tools/skills necessary to complete the online requirement. Any students who do not have access 
to a computer must be identified and supported so that they can be successful in the course. 

Successful online delivery requires that the instructor have access to some type of course 
management system such as Blackboard. The instructor must spend significant time prior to the 
start of the course to make sure that the course management site is well organized for the 
specifics of the course. Online course components include: 

1. Loading course specific documents (such as readings, homework solutions, assignments, 
syllabi) onto the course management account. 

2. Providing links to articles, podcasts, and webcasts which are required for the course. 

3. Generating online quizzes/exams which are resident on the course management system. 

4. Supporting threaded discussions on specific topics with such discussions assessed via a 
rubric. 

5. Requiring students to submit written work via the course management system. 

6. Utilizing an online homework product linked to the textbook which provides immediate 
feedback to students. 

The use of an online component in a course requires a significant initial time commitment 
on the part of the instructor and does involve some time spent during the semester 
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troubleshooting student technological issues. But once students are comfortable with the course 
details and the technology, the online components of the course will typically run very smoothly. 
Students quickly become independent in their use of the online material and are very pleased that 
this material is web-based so that it is accessible from any computer anywhere at any time of 
day. Although most students are very self-directed with respect to online material, it is useful for 
the instructor to set up the course management system so that patterns of student access and use 
are visible to the instructor and to make students aware that their patterns of online behavior are 
visible. 

LIMITATIONS 

As with any study, ours does contain some limitations. The primary one, given 
the dynamic nature of individual classes, is that the generalizability of these results 
should be interpreted cautiously. In addition, the incorporation of the Blackboard course 
management tool in the hybrid option could have influenced the outcome of student 
learning seen in our study since it provided convenient access to course materials. 

Finally, although the GPAs of the student groups were not statistically different, students 
who like to use technology in their courses could have selected the hybrid delivery and 
students who do not could have selected the fact-to-face delivery making the two groups 
different when considering characteristics other than GPAs. Our study does, however, 
provide a foundation for further research into the increasingly popular use of the hybrid 
learning option in higher education. 

CONCLUSION 

This study seeks to evaluate whether students can be equal or more successful 
academically in a hybrid structure compared with a traditional face-to-face learning 
environment, with success measured by end of term student grades. The two groups of 
students compared were enrolled in the same introductory management course taught by 
the same instructor at the same institution, a Mid-Atlantic School of Business, one year 
apart. End of term grades are used as a measure of the level of student retention of course 
material and are compared across the two groups. Analysis of the data revealed that 
student grades were significantly higher in the hybrid option than for the traditional face- 
to-face format. Results, even after controlling for gender and student ability through the 
use of GPA’s, indicate these had no significant effect on our findings. The anecdotal 
comments provided by the students support other studies that suggest high levels of 
student satisfaction are associated with online and hybrid class structures (Hall, 2006). 

In addition, we offer some pedagogical suggestions for implementation of the hybrid 
option in a college course. 

Our findings add to the current body of literature focused on understanding the 
benefits of the hybrid model and its applicability for use in higher education. In addition, 
our study assesses the value of the hybrid option in an introductory business course for 
undergraduate students. Even though some studies have been undertaken on business 
courses (Terry N. , 2007; Hall, 2006), these have been conducted on graduate level 
courses. Ours is one of the first to investigate the degree of retention for hybrid programs 
at the undergraduate level. 
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Technology has become a standard component of higher education and its use 
continues to grow and evolve as it changes the face of education. Being aware of the manner in 
which to best employ the use of technology in course design is an important aspect of being an 
educator in today’s environment. There does not appear to be a simple formula that can be 
applied to course design or specifically, how much online versus traditional face-to-face 
instruction is the optimal mix. Even experienced professors struggle with defining the proper 
balance between classroom and online components. Research in this area provides educators 
with insight into the optimal manner of delivering course material in order to maximize student 
learning outcomes. 


Applying a hybrid model, Page 8 



Journal of Instructional Pedagogies 


Volume 17, November, 2015 


APPENDIX 


Table 1 


Comparison of End-of-Course Grades in the partial on-line and traditional in-class only sections 


End-of-course grades 


Hybrid 

in = 50) 
Mean SD 

0.831 0.097 


Face-to-face 

in = 50) 

Mean SD t df 

0.782 0.095 -2.553 98 


P 

0.12 


Table 2 

Comparison of GPA’s in the partial on-line and traditional in-class only sections 

Hybrid Face-to-face 

in = 50) in = 50) 

Mean SD Mean SD t df p 

End-of-course grades 3.02 0.523 2.96 .556 0.598 98 0.248 

Table 3 


Samples of response to how the on-line portion did or did not benefit student ability to leam the 

course material 



this course is great to offer as partially online since it was a theory course, it worked because 
we had the guidance when we needed it 


on-line course information gives you the chance to go back and review chapters, PowerPoint 
presentations were very helpful in understanding class discussions 


helps with management and staying fresh with course material 


it gave us the opportunity to learn on our own 


1 could complete the quizzes and assignments when it was right for me, when 1 didn't have to 
rush and could take my time 


1 liked being able to take a quiz anytime 1 wanted and felt 1 was ready, having all the 
resources available to me was a good thing 


yes because one day a week you had to teach yourself and learn things on your own 


it gave me the option to learn when 1 felt like learning 
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